Tiingo static and fundamental data provider
Author: Eugene
Creation Date: 7/1/2017 6:02 AM
profile picture

Eugene

#1
MS123 LLC is proud to release a new static and fundamental provider delivers historical data for 37,000 U.S. stocks and ETFs from Tiingo. Like Yahoo!, it comes with a history of splits and dividends which can be applied on-the-fly and its data doesn't require refresh after a split or dividend.

Before using it you must register a free account at https://api.tiingo.com and then type in your API key in Data Manager > Tiingo

The provider is available for installation from our website (Extensions section):

Tiingo provider - installation link

Please read the online setup guide before using:

Tiingo provider online notes in the Wealth-Lab Wiki
profile picture

Eugene

#2
ReneW wrote in a thread that existed:

I really appreciate the new tiingo data provider.

The problem is, I have inserted my username and password
and still getting the error below.
I am pretty sure, that the password is correct.

I tested my account with the help of a little python script, where I need the api token,
and that worked.
Whats the problem here ? I would assume, my credentials, but I triple checked it.

QUOTE:
[3] BZQ Error: Der Remoteserver hat einen Fehler zurückgegeben: (401) Nicht autorisiert.
[0] ASHR Error: Der Remoteserver hat einen Fehler zurückgegeben: (401) Nicht autorisiert.
[2] BOIL Error: Der Remoteserver hat einen Fehler zurückgegeben: (401) Nicht autorisiert.
[1] BIB Error: Der Remoteserver hat einen Fehler zurückgegeben: (401) Nicht autorisiert.
[4] CEFL Error: Der Remoteserver hat einen Fehler zurückgegeben: (401) Nicht autorisiert.



Addon:
The quandl data provider uses the API token.... I am wondering, why accessing tiingo
has beeen realized with username/password combination instead with the API token.
profile picture

Eugene

#3
It's really unfortunate that it doesn't work for you. I have no problem downloading data from Tiingo:

QUOTE:
Updating DataSet 1 ...
[1] CEFL 899 bars 02.08.2017 899 bars added
[0] ASHR 941 bars 02.08.2017 941 bars added
[3] BOIL 1465 bars 02.08.2017 1465 bars added
[2] BIB 1843 bars 02.08.2017 1843 bars added
[4] BZQ 2045 bars 02.08.2017 2045 bars added
Update completed (2.918 sec)
----------------


Let's collect some evidence first:

1. Are your credentials from api.tiingo.com and not from some other part?
2. They are case sensitive. Have you double checked that they're entered exactly?
3. Can you log in on their API website successfully?
4. How many bandwidth and requests are left: Your API usage?

I'll mark to investigate into Auth Token authorization later this or next month but basic HTTP authorization works for me.
profile picture

ReneW

#4
QUOTE:
Let's collect some evidence first:



1.) https://api.tiingo.com/
2.) checked
3.) yes
4.) I downloaded only the ticker SPY as a test case, enough bandwith left


SOLVED:

I can login on tiingo with <username>@<email.com>

_BUT_ WL data provider works only with username <username>

Sorry!

profile picture

Eugene

#5
Okay, it was a basic check but still. Basic HTTP authorization the provider uses is so simple that there's hardly a place to fail. Maybe your username has some special characters? Or this might be something related to our different regional settings etc. It would be very hard to tell the reason given that it's impossible to reproduce.
profile picture

Eugene

#6
QUOTE:
I can login on tiingo with <username>@<email.com>

_BUT_ WL data provider works only with username <username>


Glad you solved it while I was replying :-)

Username must be username, not email or anything. It's not the "WL data provider" but it's precisely the Tiingo API that accepts an account name and does not accept an email.

I've added a note to the Wiki page. Thanks.
profile picture

Eugene

#7
Tiingo provider updated to v2018.01:

* Fixed error message on broken config file
profile picture

Panache

#8
I'm running into the same issue as ReneW using Tingo v. 2018.01.

1.) https://api.tiingo.com/
2.) checked
3.) yes
4.) I downloaded only the ticker SPY as a test case, enough bandwith left

I am only using my Tiingo Username (ie. not my email address) and Password.

QUOTE:
Updating DataSet Tiingo ...
[0] SPY Error: The remote server returned an error: (401) Unauthorized.
Update completed (0.336 sec)
----------------
profile picture

Eugene

#9
Cannot reproduce:
QUOTE:
Updating DataSet 1 ...
[0] SPY 6297 bars 30.01.2018 6297 bars added
Update completed (27.315 sec)
----------------
profile picture

Revilla

#10
Hello support team,

I have the Extension Tiingo Static, Real-Time and Fundamental
installed by MS123 (Eugene) in WL.
Unfortunately, I do not get any data from Tiingo. I get the message that I am not authorized.
On request to Tiingo I got the following answer.

"Hi Egbert,

WealthLab's software but it sounds like you may need your API authorization token! I'm guessing the software has a place where you can input your token.

Your API token is: cad69c8e....................

And going forward you can find your token here: https://api.tiingo.com/account/token

Let us know if this helps
Rishi "

Where can I insert the token in WL?

Best regards
Egbert
profile picture

Eugene

#11
Hi Egbert,

Please please always read the online user guide before using and/or asking questions to both support teams. You'll find your question answered in at least four places:

1. If you try to update Tiingo data not having entered a username/password, you're going to see a warning message in Wealth-Lab's "Data Update Log" view followed by explicit instructions on where to insert the password.

2. Straight in the first post of this topic (see notes in bold)

3. On the 2nd screenshot of the extension download link from where you installed it

4. And of course in the online user guide at: Tiingo Static Provider

P.S. Tiingo's API token has no use in WL. The provider uses the password from your account.
profile picture

Panache

#12
Eugene,

I'm still getting the same error message reported in post #8.

I've tried everything I can think of, including deleting TiingoClientSettings.xml and copying and pasting my username and password into the Tiingo tab in DataManager. I checked the new TiingoClientSettings.xml file and both the username and password are correct.

I can copy and paste the same username and password and get logged in at https://api.tiingo.com/. Unlike ReneW, my username is not associated with an email address, but is a random combination of 12 upper and lower case letters and numbers. My password also is a random combination of 12 upper and lower case letters and numbers.

Any thoughts as to what I might be doing wrong?
profile picture

Eugene

#13
Kurt,

It still works for me but both my login/password are shorter and lower case (since I only need them for testing). You might want to try a shorter and simpler account name AND password in lower case, or create a support ticket to pass on your credentials to me to check what goes wrong. If all else fails I'd have to experiment with replacement of Basic authorization with Token authorization.
profile picture

Eugene

#14
P.S. Or you can simply wait until the end of month when we release an update to Tiingo provider which I hope will fix this for everyone. This is going to be a breaking change: current basic authentication using login/password has proven unreliable and will get replaced with token authentication using the API key from Tiingo's website. Consequently, username/password will stop working starting from v2018.03 in March and to update any data, Wealth-Lab users will have to obtain the free API key and paste it into the Data Manager.
profile picture

Eugene

#15
What's new in v2018.03:

This version contains a breaking change. If your current version (2018.01 or 2017.07) still works for you, you may not update until you get an error on data update like Error: The remote server returned an error: (401) Unauthorized.

* Change: BREAKING CHANGE: switched to token authentication; must copy/paste your API Token from api.tiingo.com website before using

We noticed that Tiingo no longer features Basic authentication (login/password) in their API documentation pages and it fails for some WLP/D users. To resolve this issue we have switched to Token authentication. For this reason, starting from v2018.03 your saved login/password will no longer work. To make Tiingo update data again you'll have to obtain your free "API Key" at api.tiingo.com and copy/paste it (no typing !!!) into the Data Manager (Tiingo tab).
profile picture

Panache

#16
Thank you Eugene. At least for my initial DataSet just containing SPY, it works as expected with 6316 bars added.
profile picture

Eugene

#17
Kurt, glad to have helped.
profile picture

Eugene

#18
What's new in v2018.04:

* New: support for GetSessionOpen

Tiingo's EOD data does not contain the partial bar, limiting its application in Strategies which require knowledge of the open price for today's incomplete bar - like gap trading, for example.

The good news is that their IEX Real-time Data API has recently become free for use by individuals. We've plugged it in to bring support for the open price for "today".

Since the liquidity is less than other exchanges, we recommend using GetSessionOpen for large cap companies and ETFs. Price updates can be less frequent for small cap companies.
profile picture

Eugene

#19
What's new in v2018.12:

Fix: potential error on data update in the Strategy Monitor only (The added or subtracted value results in an un-representable DateTime).
profile picture

Panache

#20
What are Tiingo split values? Specifically, how do I apply
CODE:
Please log in to see this code.
to the raw data to get split-adjusted Tiingo data? With Fidelity, Wealth-Data and Yahoo splits you can just divide OHLC by the split value and multiply V by the split value, but with the Tiingo numbers, that isn't even going to be the right order of magnitude.

CODE:
Please log in to see this code.
profile picture

Eugene

#21
Here's my output of FundamentalDataItems("Split (Tiingo)"):
CODE:
Please log in to see this code.

Everything looks correct. Try to reload the data or delete the TiingoFundamentalProvider folder.
profile picture

Panache

#22
My apologies. I had reloaded Tiingo into newly created TiingoStaticProvider and TiingoFundamentalProvider directories to avoid any problems with old data.

I just reran my code and am now getting the same thing you get. I have no idea what happened yesterday.
profile picture

Eugene

#23
Glad it works for you.
profile picture

Panache

#24
I can't update any data from Tiingo. Please see the attached update log. (For the sake of saving server space, I deleted everything after the first three symbols, since it is just the same error for every symbol.)
profile picture

Eugene

#25
QUOTE:
Invalid JSON primitive: You.

My gut feel is that server is trying to tell You something re: its usage limitations. You should find an answer at the bottom of the online guide > Issues and Limitations.

Of course the handling of the error message should be more graceful. Something to think about for a future release.
profile picture

Panache

#26
QUOTE:
usage limitations
You are absolutely right. Apparently my memory is failing me, because I remembered it as being 500/day.
profile picture

Eugene

#27
FYI, in addition to this error "Data parsing error. Symbol: xxx, LineNumber: -1, String: Message: Invalid JSON primitive: You..." you may also encounter a HTTP error code 429 error message. Sounds cryptic too and actually indicates that a free API user has exceeded the number of requests in an hour (500 items).
profile picture

Panache

#28
QUOTE:
My gut feel is that server is trying to tell You something re: its usage limitations.
I ran into the:

"[1] CRRC Error: Data parsing error. Symbol: CRRC, LineNumber: -1, String:

Message:
Invalid JSON primitive: You.
Stack Trace:
at System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializePrimitiveObject()
at System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializeInternal(Int32 depth)
at System.Web.Script.Serialization.JavaScriptObjectDeserializer.BasicDeserialize(String input, Int32 depthLimit, JavaScriptSerializer serializer)
at System.Web.Script.Serialization.JavaScriptSerializer.Deserialize(JavaScriptSerializer serializer, String input, Type type, Int32 depthLimit)
at ﷓.﷐(String ﷐, String ﷑)"

problem again this morning. Attached is a screenshot of my Tiingo API usage taken right after the provider got done throwing the errors.

It does appear to be a usage issue. I put a different API Token in and was able to update the data.
profile picture

Eugene

#29
Perhaps an issue with your account quota at Tiingo end. Yes, it's still the same error as in post #25. I also ran into this limitation during testing and confirmed that the message starts with "You..." (can't remember the full text). One day I may be able to make it read more clear rather than cryptic but it's a low priority task that does not require immediate resolution.
profile picture

Panache

#30
QUOTE:
if Tiingo already works and is reliable
In my humble opinion, Tiingo data is about the same quality as Yahoo.

Also, Eugene, just a reminder that the way the Tiingo Provider is coded, Tiingo thinks you are using a lot more data than what you actually get.
profile picture

Eugene

#31
What's new in v2020.09:

* New: Added support for intraday data (1-min to 60-min)
* New: Added Streaming data provider
* New: Added support for Strategy Monitor
* New: Added Paper trading support

Compatible with WLD 6.9.23 and up.
profile picture

Harapa

#32
Thanks for adding intraday option. Look good.
profile picture

Eugene

#33
Glad you like it. Please note that the U.S. intraday data at Tiingo comes from the IEX exchange and may not match the official data exactly.
profile picture

thodder

#34
I updated the provider today and my Tiingo datasets were no longer displayed in WLD. I realized this when I did an Update All Data and it didn't do anything...

Message:
Updating Provider Tiingo ...

Provider update complete for Tiingo
----------------


WLD version 6.9.23.1 Tiingo provider: 2020.09
profile picture

Eugene

#35
Please try the following:

1. Close WLD
2. Delete your TiingoStaticProvider folder together the XML file inside

c:\Users\Windows username\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\TiingoStaticProvider

3. Enter your API key
4. Update
profile picture

thodder

#36
Nope. I can create a new dataset for Tiingo and I'm able to updated it fine, but Tiingo datasets which existed prior to the latest update no longer display in the Datasets list.

Attached is a zip file with two datasets: "Tiingo Test" created after the latest update whereas the other one was created before.
profile picture

Eugene

#37
Please re-create your old Tiingo DataSets. The new provider version has probably lost its backward compatibility with them due to the many changes. In particular, the XML file's definition has been extended a bit. Sorry for the inconvenience.
profile picture

lkirsch

#38
My Tiingo data is not updating after 8/31 Is there a bug in the new code?
profile picture

thodder

#39
I had the same problem. The Tiingo datasets didn't even show in the Dataset list. I had to rebuild those datasets. It involved:

1. Close WLD
2. Delete (or move) each Tiingo dataset from the Datasets folder

c:\Users\Windows username\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\Datasets

3. Run WLD
4. Create datasets you just deleted

I could easily rebuild my datasets so I just deleted them. If you need to copy what was in the old dataset, you should move them (in step #2) to a temporary folder; then you'll need to peek inside the Tiingo dataset with a utility that can read an xml file.

A bit of a nuisance, but Tiingo provider seems to work better than before once I rebuilt the datasets.
profile picture

wl64bit

#40
I followed Eugene's steps in post #34, but the last bar data is still being shown as of 8/31/2020.
profile picture

thodder

#41
Last bar for me is 8/31/2020 too. I hadn't looked close enough at the data after getting the latest update.

I couldn't find anything on Tiingo's web site to request a quote to see if it was their server or WLD provider.
profile picture

Eugene

#42
QUOTE:
My Tiingo data is not updating after 8/31 Is there a bug in the new code?

Make sure your API key is present on the Tiingo tab. If not, please reenter it.

QUOTE:
Last bar for me is 8/31/2020 too.

Works fine for me here:

Updating DataSet Tiingo (Daily) ...
[0] AAPL 10015 bars 01.09.2020 10015 bars added
Update completed (4.336 sec)
----------------

QUOTE:
I couldn't find anything on Tiingo's web site to request a quote to see if it was their server or WLD provider.

Tech savvy users could log in to api.tiingo.com in the browser and try making a query to verify the data is actual or not. Right now it returns an error (502 Bad Gateway) so maybe they're having issues.
profile picture

MrIzzy

#43
Hi There,

I am trying to use Tiingo with no success at all
1) Registered for an account
2) Installed the Provider Extension Tiingo 2020.09
2) Copied the AuthToken into WLD (under the Tiingo) tab
3) I have three symbols in a Daily data set: IBM, AAPL, TSLA
4) Even after following post #38, I get




Daily data is being updated...

Updating Provider Tiingo ...

[2] TSLA Error: The remote server returned an error: (404) Not Found.
[1] IBM Error: The remote server returned an error: (404) Not Found.
[0] AAPL Error: The remote server returned an error: (404) Not Found.
Provider update complete for Tiingo
----------------


Update completed (0.235 sec)
----------------

profile picture

Eugene

#44
Like I just said, right now the server returns an error (502 Bad Gateway) so they're having issues.
profile picture

lkirsch

#45
Eugene:

I just updated AAPL and I get:

[0] AAPL 10015 bars 8/31/2020 10015 bars added

Notice the number of bars is the same as you reported in #41, but the last date is 8/31. What is your last date?
profile picture

wl64bit

#46
I am having exactly the same issue as lkirsch: the last date is 8/31/2020
profile picture

Eugene

#47
At the risk of repeating the 3rd time in 3 hours: the Tiingo API must be having issues.
profile picture

wl64bit

#48
Tiingo does not have any issues at this moment. It has to do with the new Static Provider. Here is the quote from the Tiingo support:
"rsingh [21:14:54]: Yeah not an issue on our end, saw your E-mail. I would reach out to the maintainer of the program".

Now it updated the dataset with the date 9/1/2020, while it should be 9/2/2020.
profile picture

Eugene

#49
Works fine here

Updating DataSet Tiingo (Daily) ...
[2] TSLA 2564 bars 02.09.2020 2564 bars added
[1] QQQ 5408 bars 02.09.2020 5408 bars added
[0] AAPL 10016 bars 02.09.2020 10016 bars added
Update completed (8.613 sec)
----------------
profile picture

rmandel00

#50
There is still a problem with Tiingo. My data sets updated only one day, up to 9/1/20. If I check the SPY OHLC from 9/1/20 it is identical to the Yahoo data for 9/2/20. If I examine the Tiingo data day by day there is data on 8/30/20 which is a Sunday. This data corresponds to Yahoo data from 8/31/20. As a result of using Sunday as a trading day, the Tiingo data is one day off.

I deleted the data set, recreated it and it already showed the same data as before, one day off.

Any suggestions?
profile picture

Eugene

#51
I'm using the latest provider version. Works for me here in Europe. Maybe you U.S. guys are hitting some bad server at the data farm, don't know.

Just made the Data Tool run a data integrity check on a freshly reloaded Tiingo EOD DataSet with SPY and it had no errors and zero issues in the output log. The data is visually equal to Yahoo's, updated to 9/2/2020 and without weekend bars (which the Data Tool would've caught). Try to refresh the data (please see the WL User Guide on how to reload chart history, or Data Tool's online manual if using it).

QUOTE:
I deleted the data set, recreated it and it already showed the same data as before, one day off.

Recreating a Dataset is harmless but useless. It's akin to deleting and recreating a Windows shortcut.
profile picture

Harapa

#52
I have about 6 data sets for this provider. When I try to update using 'update all data for the provider', it updates only 3 of them ( alwaya the same three), however, I can manually update all (thus quota is not the issue).
profile picture

thodder

#53
You may be onto something rmandel00. I was able to run the Data Tool - Data Checker. I checked off: missing dates, synthetic bars and weekend bars. When I ran it against a dataset with 23 symbols I got a lot of messages stating "weekend bar" and some "Synthetic bar" - 14,507 lines of errors to be exact. To avoid cluttering the post, I attached a text file with the errors.

This is a new dataset that I created on Tuesday (9/1/2020) plus I deleted my TiingoStaticProvider directory as Eugene requested on post #34. So this is all fresh data.
profile picture

Eugene

#54
@Harapa

1. How can this be reproduced? Also, can you include the data update log?
2. Have you deleted and recreated the old DataSets after updating the provider?

@ thodder

Updated the DataSet with a handful of symbols (AVGO CIEN CLDR WUBA ZUO) from your log. No errors in the Data Tool checker's log.

Perplexing but still works for me. Perhaps the U.S. settings or time zone may be the thing that makes the difference. I'll check the code again. :/
profile picture

Harapa

#55
1. Don't know how to reproduce on a different computer? Here is the log of update all data set.
QUOTE:
Updating Provider Tiingo ...


5 Minute data is being updated...

[2] DIA 322922 bars 9/3/2020 22 bars added
...

30 Minute data is being updated...

[0] AAPL 10000 bars 9/3/2020 0 bars added 1 bars corrected
...

60 Minute data is being updated...

[0] AAPL 6458 bars 9/3/2020 1 bars added
...

Provider update complete for Tiingo
----------------

Update completed (26.442 sec)
----------------


2. I did that.
profile picture

Eugene

#56
And what should this log tell me? WL correctly updated 5, 30 and 60 min data up until today. OK with me. There is no concept of DataSets in "Update All Data" mode.

If you're having what you think is a problem, please explain it clearly, with as many evidence as you can think of.
profile picture

KGo

#57
I created a Tiingo dataset a few days ago. Tiingo dates are one day earlier than the prices. So the Tiingo prices on 4/15/2010 are shown on 4/16/2010 with Yahoo and Fidelity. Prices may not be exactly the same but the shape of the candles clearly show the same candle with different dates. This is consistent for all symbols. Updates are more of the same.

The update log is unremarkable. The last date seems one day off.
profile picture

Eugene

#58
@KGo, rmandel00, wl64bit, lkirsch, thodder

Re: data appears not up to date in the U.S., a.k.a. delayed timestamp (one bar off), and weekend bars etc. Bug fixed. Please update to v2020.09.3. Thank you all for reporting it and sorry for the inconvenience.

Reload your already downloaded data! It's timestamped incorrectly. To do that, delete the TiingoStaticProvider folder (except the TiingoClientSettings.xml file which contains your API key) and update data.
profile picture

Harapa

#59

As I said I have more than 3 data sets for Tiingo provider.
When I go to 'Update data' tab, select Tiingo, and click on Update data for selected provider, it only updates the three I showed above and not the rest, though I can manually update the rest of data sets. Shouldn't this routine update all Datasets for this provider? [ As is observed for all other providers].
Here are the headers of manually updating the left outs, all of which were successful.
QUOTE:

Updating DataSet AMAGX (M60) ...
Updating DataSet AMAGX (M30) ...
Updating DataSet AMAGX (M60) ...
Updating DataSet Tiingo (NAS100M30) ...
profile picture

thodder

#60
On Eugene's post #57, look at the steps in post #34 for reloading data as you will lose your API key when you delete the TiingoStaticProvider folder. I cheated and deleted the contents of TiingoStaticProvider except for the xml file which contains the key.
profile picture

Eugene

#61
@Harapa: Try to delete your DataSets and recreate them.

@thodder: thanks, updated post #57.
profile picture

KGo

#62
Eugene,
After installing new Tiingo provider, deleting the static data and updating Tiingo data sets all bar history was downloaded correctly dated.

Thanks for the quick fix!
profile picture

thodder

#63
Eugene,

I'm running into a similar issue as Harapa, but I'm not sure it's an issue with only the Tiingo provider; however, I did this test with only Tiingo data.

1) I have three Tiingo datasets. Prior to running an "Update All..." i manually updated prices on each of the three datasets and used the Data Tool to check that all symbols had data. They did.

2) I ran "Update All.." with only the Tiingo provider checked. The option "Delete data for Symbols that are not contained in DataSets" is checked. The attached log gives a lot of status error 429 -- I believe that's because I've run over my free quote allowed after running multiple times today. My concern is the "Deleted symbols" at the end of the update.

3) Using Data Tool: one Tiingo dataset has data; another all data is gone; the last one only half the symbols have data. In step #1 I used the Data Tool to make sure all datasets had data. That means the "Update All..." removed symbol data when there were datasets containing those symbols.

I have 9 providers installed with datasets using 4 of those providers. This test was only for Tiingo, but I think I've seen similar results with Yahoo.

Let me know if you need more info to follow up on this.

UPDATE: It looks like the log attachment didn't upload. I've overlayed it already. Because I keep getting the 429 error now I can't recreate the test today.
profile picture

Harapa

#64
I recreated all data sets. Still only 3 data sets are updated at a time.

QUOTE:
There is no concept of DataSets in "Update All Data" mode.

I understand, in this mode all symbols are pooled and than updated. But in "Tiingo" case it is not pooling the symbols in different data sets. For example, I have three different data sets of 5 minutes, of which only one is updated in "Update All Data" mode.

Edit: I confirm ( as noticed by @ ashridharani) the data sets being updated are the last alphabetically listed Tiingo dataset. Also only one dataset for a given bar interval is updated.
profile picture

ashridharani

#65
I ran into similar issue as Harapa after updating the Tingo Extension following the directions in post #34 and #57. I download 60 unique symbols daily, spread across 10 differently named datasets. Quiet a few of these 60 symbols appear in a multiple of those 10 datasets. When I ran the Data Manager for daily update (I only need daily prices) only 35 symbols would get updated. No matter how many times I tried, I would get the same 35 symbols only. I noticed that those 35 symbols corresponded to the very last alphabetically listed Dataset of the 10 Tingo based datasets listed in Data Manager. As an experiment I copied the remaining 25 symbols into the last dataset and ran the Data Manager again. Voila, now I had all the 60 symbols data. To me, in my case, it looks like the Tingo Extension is only looking at one Dataset (which happens to be the last alphabetically listed Tingo dataset) and updates symbols only within that dataset, unlike the previous version whereby it would go through all the datasets and pool all the unique symbols to update them. Harapa you may want to try this and see if works for you.
profile picture

Eugene

#66
@ashridharani

Thanks for the detailed bug report, I will look into it today.

It's very odd since I specifically tested the scenario "Update All Data for multiple DataSets in several time frames".
UPDATE: Even though I did, the choice of symbols was not sufficient to find this bug. Working on a fix.
profile picture

Eugene

#67
@Harapa, ashridharani

Re: symbols skipped, only last DataSet is always updated in "Update All Data" mode. Bug fixed. Please update to v2020.09.4

Thank you all for reporting it and sorry for the inconvenience.
profile picture

Harapa

#68
It is good now. Thanks for the help.
profile picture

thodder

#69
Thanks Eugene! The latest fixes correct all my issues. Thanks for the quick turnaround.
profile picture

Harapa

#70
Recent splits of AAPL and TSLA are not being applied to intraday data (5,30, 60 minutes). I have split/dividend checked.
Removing/deleting old data file & updating or reload chart history, or manually apply Stock split...doesn't work.
profile picture

Eugene

#71
It's OK. Tiingo's intraday data does not contain splits. I've just added a note to the online user guide. And AFAIR, WL's "Apply stock split" applies to EOD data only.
profile picture

Harapa

#72
Thanks for clarification.
profile picture

rmandel00

#73
I was looking over some of the Tiingo Data downloads to WLD and notice that there is a problem with SPY. It appears that Friday data is not there. Fore example if I open a chart of SPY i have no bar on 8/7/20, 8/14/20, 8/21/20. This is pretty consistent going back in time. For example 3/13/20, 3/20/20 and 3/27/20 are missing and so forth. I need to us the SPY in my strategies. Do you have a suggestion or a fix.
Thanks,
RichMan
profile picture

Eugene

#74
Data Checker's report (by the Data Tool) shows no missing dates in SPY. Therefore in accordance with post #57 above, two suggestions:

1. Update the extension.
2. Reload the data.
profile picture

rmandel00

#75
Eugene,
Thanks.
I found that I had to delete the appropriate .WL files in the Tiingo Fundamental Provider folder and then reload the data.
That worked.
Rich Man
profile picture

Eugene

#76
Glad to hear it works for you. However, fundamental data has no effect on the missing bars and is reloaded automatically on each data update.
profile picture

rmandel00

#77
You are of course right. It was the Tiingo Static Provider folder that I deleted the symbols from.
profile picture

KGo

#78
When using "Perform split adjustment" the Tiingo provider is not splitting dividends. Needs a fix.

Otherwise this is an excellent provider. Dividends are often made available here quickly and they are captured with extended digit accuracy as they were originally declared. Special dividends are given that cause discontinuity in price series but are typically correctly captured. Below is sample split dividend calculation applied to Tiingo. The adjusted Tiingo matches Wealth-Data and the Yahoo dividend is called out for possible error which can be seen in the last 2 dividends which were not split. The Tiingo behavior is consistent across all symbols.
QUOTE:
TMV
Dividend (Tiingo) 6/25/2019 0.7038 * 0.1 =Orig Value of 0.07038
Dividend (Tiingo) 9/24/2019 0.5518 * 0.1 =Orig Value of 0.05518
Dividend (Tiingo) 12/23/2019 0.21 * 0.1 =Orig Value of 0.021
Dividend (Tiingo) 3/24/2020 0.202 * 0.1 =Orig Value of 0.0202


Dividend (Wealth-Data) 6/25/2019 0.7
Dividend (Wealth-Data) 9/24/2019 0.55
Dividend (Wealth-Data) 12/23/2019 0.21
Dividend (Wealth-Data) 3/24/2020 0.2


Dividend (Yahoo! Finance) 6/25/2019 0.7
Dividend (Yahoo! Finance) 9/24/2019 0.55
Dividend (Yahoo! Finance) 12/23/2019 0.021
Dividend (Yahoo! Finance) 3/24/2020 0.02


Tiingo TMV -117.8200 1.6676 -116.1524
Wealth-Data TMV -118.0700 1.6600 -116.4100
Yahoo TMV -117.8200 1.2910 -116.5290


TMV Yahoo dividend


Split (Tiingo) 4/23/2020 0.1


Split (Wealth-Data) 4/23/2020 0.1


Split (Yahoo! Finance) 4/23/2020 0.1
profile picture

Eugene

#79
@Panache

QUOTE:
Also, Eugene, just a reminder that the way the Tiingo Provider is coded, Tiingo thinks you are using a lot more data than what you actually get.


Unless you use Tiingo for streaming charts/quotes and/or on demand data update is enabled, the provider does not make any superfluous requests. Otherwise please clarify your point.
profile picture

KGo

#80
Bump Post 78 dated 9/30/2020

QUOTE:
When using "Perform split adjustment" the Tiingo provider is not splitting dividends. Needs a fix.


TMV was cited above. The Tiingo dividend in the WLF needs to be adjusted for splits. Best I can tell, it seems every split Tiingo symbol has the same issue. Including::
AAPL,ABT,ADBE,ADSK,AIG,AMLP,ASML,BAC,C,CAT,CL,CMCSA,COP,CSX,CVS,CVX,DHR,DUK,DUST,EEM,EFA,EMR,EPP,EWI,EWJ,
EWM,EWS,EWT,EWU,EZU,FAS,FAST,FAZ,FXI,GD,GDXJ,HON,IBB,IJH,IJR,IWM,IWN,IYR,JDST,JNK,JNUG,KO,LOW,MA,MAR,MET,NEE,NKE,
NTES,NUGT,OIH,OXY,PCAR,PFE,PG,PSQ,QCOM,QID,QLD,REM,RING,ROST,RTX,RWM,SBUX,SDS,SH,SLB,SPXL,SPXU,SQQQ,SSO,TBT,
TCOM,TMF,TMV,TNA,TQQQ,TWM,TZA,UNH,UNP,UPRO,V,VOO,VTI,VWO,WFC,XOP,XRT

Do you see this? Or have I something wrong?
profile picture

Eugene

#81
We will have to live with this peculiarity.
profile picture

KGo

#82
QUOTE:
We will have to live with this peculiarity.


OK but let's be clear. It is a bug that results in incorrect dividends. Any backtest on AAPL for example will show dividends as 400% to 2800% of what were actually paid. Results are not just "peculiar" (which seems to imply odd but ok) they are flat out wrong. And it leaves all customers using Tiingo, a nice low cost provider, with no way to fix the incorrect results.
profile picture

Eugene

#83
You can use "Apply dividends to backtest results when using Portfolio Simulation mode", make sure to read the User Guide > Preferences > Backtest Settings.
profile picture

KGo

#84
QUOTE:
You can use "Apply dividends to backtest results when using Portfolio Simulation mode"

Sorry but that's what gives the wrong answers. Dividend (Tiingo) applies dividends from the wlf that are not split. Dividend (Yahoo) and Dividend(Wealth-Data) apply correctly to their respective data.

From Post 78 example a performance report with dividends applied for
Tiingo 1 share hold over 6/25/2019 shows 0.07 dividend and
Wealth-Data 1 share hold over 6/25/2019 shows 0.70 dividend at the same per share price.

The difference is that Tiingo has not adjusted for the 0.1 split factor. Please recognize that every Tiingo split dividend is wrongly applied as with TMV. I have written a program to compare dividends across 3 providers and Tiingo values require split adjustment to match Wealth-Data and Yahoo.

I have attached 1800 dividend examples like this. The attached shows the Tiingo value AFTER the given split factor was applied. The close match to Yahoo and Wealth Data is evident. However it is incredibly easy to test 1 share to see the issue in a performance report for any of the attached dividends.
profile picture

Eugene

#85
QUOTE:
Dividend (Yahoo) and Dividend(Wealth-Data) apply correctly to their respective data.

They can also be applied to Tiingo data.
profile picture

KGo

#86
QUOTE:
They can also be applied to Tiingo data.

Yes WD and Yahoo could be applied to Tiingo price data and sometimes the results will be correct.

But when provider prices do not match nonsense results. Tiingo may record a $9.00 dividend when WD or Yahoo record a split with $0.00 dividend. This throws that and every prior dividend value out of price alignment and makes mismatching price provider and dividend provider unacceptable.

The Tiingo provider should really be fixed.

profile picture

Eugene

#87
Thank you for pointing this out. I've just ensured that Tiingo dividends will be split-adjusted correctly in WL7's upcoming provider extension.

Sorry but WL6 users will have to live with this as we no longer update the WL6 extensions.
profile picture

eralbanese

#88
Hello,

I have the tiingo data feed (upgraded) and cannot get WLD to backtest beyond 1/1/2021 on the one minute time frame. Is this something to do with tiingo’s feed or with my setup? I apologize if this has been asked before but I did not see it in a search.

Thanks
Eric
profile picture

Eugene

#89
Hi Eric,

According to this page the IEX intraday data is available since 2017 so I'd suggest you check your data loading range in WLD. If it's like "All data" and no instruments history goes beyond 2021, Tiingo should know best.
profile picture

eralbanese

#90
Eugene,

From Tiingo Support
QUOTE:
To get data in the past you must pass both a start date and end date with the API. Does wealthlab give you the ability to pass both a start date and an endDate? If so, can you set the start date to 2020-01-01 and end Date to 2020-02-03 as a test?


Thanks,
Eric
profile picture

eralbanese

#91
As a follow-up from Tiingo
QUOTE:
Let them know the API returns 10k bars in each query. So if you want data prior to then, Start dates and end dates must be passed explicitly. E.g. https://api.tiingo.com/iex/aapl/prices?startDate=2019-12-1&endDate=2020-01-01&resampleFreq=1min


He also gave me a direct email if you'd like.
profile picture

Eugene

#92
Eric, thank you for the follow-up. WL6 does pass a start and end date but we weren't aware of the 10k bars / query limitation. It's undocumented in their API user guide.

We will implement pagination of intraday requests in the upcoming Tiingo data provider for WL7. It will be part of a new paid extension for WL7 called DataExtensions which contains several data providers, indicators that download and cache data, demo strategies to illustrate their usage etc.
profile picture

jotapoker

#93
Tiingo has been working great for the Daily and 60min timeframes. But when i try to use smaller timeframes (like the 15min) i get a message sayig "no data". Why is that?
profile picture

Eugene

#94
The answer is probably right on top - please see post #92.
profile picture

jotapoker

#95
Somehow, all the timeframes are working now for stocks.

Now i'm having problems with crypto symbols. For daily timeframe i just get data starting from March 1st 2021. Other times frames i receive the "no data" message.

Any help?
profile picture

Eugene

#96
The WL6 provider will not support cryptos. Its WL7 counterpart from DataExtensions has advanced to fully support cryptos and load the entire intraday data (not a short fragment of it like in WL6).

https://www.wealth-lab.com/extension/detail/DataExtensions

I recall you were beta testing it but you can have an extended 90-day free trial of WL7.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).